4 research outputs found
What a Nerd! Beating Students and Vector Cosine in the ESL and TOEFL Datasets
In this paper, we claim that Vector Cosine, which is generally considered one
of the most efficient unsupervised measures for identifying word similarity in
Vector Space Models, can be outperformed by a completely unsupervised measure
that evaluates the extent of the intersection among the most associated
contexts of two target words, weighting such intersection according to the rank
of the shared contexts in the dependency ranked lists. This claim comes from
the hypothesis that similar words do not simply occur in similar contexts, but
they share a larger portion of their most relevant contexts compared to other
related words. To prove it, we describe and evaluate APSyn, a variant of
Average Precision that, independently of the adopted parameters, outperforms
the Vector Cosine and the co-occurrence on the ESL and TOEFL test sets. In the
best setting, APSyn reaches 0.73 accuracy on the ESL dataset and 0.70 accuracy
in the TOEFL dataset, beating therefore the non-English US college applicants
(whose average, as reported in the literature, is 64.50%) and several
state-of-the-art approaches.Comment: in LREC 201
Unsupervised Measure of Word Similarity: How to Outperform Co-occurrence and Vector Cosine in VSMs
In this paper, we claim that vector cosine, which is generally considered
among the most efficient unsupervised measures for identifying word similarity
in Vector Space Models, can be outperformed by an unsupervised measure that
calculates the extent of the intersection among the most mutually dependent
contexts of the target words. To prove it, we describe and evaluate APSyn, a
variant of the Average Precision that, without any optimization, outperforms
the vector cosine and the co-occurrence on the standard ESL test set, with an
improvement ranging between +9.00% and +17.98%, depending on the number of
chosen top contexts.Comment: in AAAI 2016. arXiv admin note: substantial text overlap with
arXiv:1603.0870
ROOT13: Spotting Hypernyms, Co-Hyponyms and Randoms
In this paper, we describe ROOT13, a supervised system for the classification
of hypernyms, co-hyponyms and random words. The system relies on a Random
Forest algorithm and 13 unsupervised corpus-based features. We evaluate it with
a 10-fold cross validation on 9,600 pairs, equally distributed among the three
classes and involving several Parts-Of-Speech (i.e. adjectives, nouns and
verbs). When all the classes are present, ROOT13 achieves an F1 score of 88.3%,
against a baseline of 57.6% (vector cosine). When the classification is binary,
ROOT13 achieves the following results: hypernyms-co-hyponyms (93.4% vs. 60.2%),
hypernymsrandom (92.3% vs. 65.5%) and co-hyponyms-random (97.3% vs. 81.5%). Our
results are competitive with stateof-the-art models.Comment: in AAAI 201
Nine Features in a Random Forest to Learn Taxonomical Semantic Relations
ROOT9 is a supervised system for the classification of hypernyms, co-hyponyms
and random words that is derived from the already introduced ROOT13 (Santus et
al., 2016). It relies on a Random Forest algorithm and nine unsupervised
corpus-based features. We evaluate it with a 10-fold cross validation on 9,600
pairs, equally distributed among the three classes and involving several
Parts-Of-Speech (i.e. adjectives, nouns and verbs). When all the classes are
present, ROOT9 achieves an F1 score of 90.7%, against a baseline of 57.2%
(vector cosine). When the classification is binary, ROOT9 achieves the
following results against the baseline: hypernyms-co-hyponyms 95.7% vs. 69.8%,
hypernyms-random 91.8% vs. 64.1% and co-hyponyms-random 97.8% vs. 79.4%. In
order to compare the performance with the state-of-the-art, we have also
evaluated ROOT9 in subsets of the Weeds et al. (2014) datasets, proving that it
is in fact competitive. Finally, we investigated whether the system learns the
semantic relation or it simply learns the prototypical hypernyms, as claimed by
Levy et al. (2015). The second possibility seems to be the most likely, even
though ROOT9 can be trained on negative examples (i.e., switched hypernyms) to
drastically reduce this bias.Comment: in LREC 201